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Abstract 

We consider the problem of self-healing in peer-to-peer networks that are under repeated attack by 
an omniscient adversary. We assume that the following process continues for up to n rounds where n 
is the total number of nodes initially in the network: the adversary deletes an arbitrary node from the 
network, then the network responds by quickly adding a small number of new edges. 

We present a distributed data structure that ensures two key properties. First, the diameter of 
the network is never more than 0(log A) times its original diameter, where A is the maximum degree 
of the network initially. We note that for many peer-to-peer systems, A is polylogarithmic, so the 
<D diameter increase would be a 0(log log n) multiplicative factor. Second, the degree of any node never 

increases by more than 3 over its original degree. Our data structure is fully distributed, has 0(1) 
fN| latency per round and requires each node to send and receive O(l) messages per round. The data 

CN structure requires an initial setup phase that has latency equal to the diameter of the original network, 

and requires, with high probability, each node v to send O(logn) messages along every edge incident 
to v. Our approach is orthogonal and complementary to traditional topology-based approaches to 
defending against attack. 

q 1 Introduction 

Many modern networks are reconfigurable, in the sense that the topology of the network can be changed 
r—i by the nodes in the network. For example, peer-to-peer, wireless and mobile networks are reconfigurable. 

More generally, many social networks, such as a company's organizational chart; infrastructure networks, 
such as an airline's transportation network; and biological networks, such as the human brain, are also 
reconfigurable. Unfortunately, our mathematical and algorithmic tools have not developed to the point 
that we are able to fully understand and exploit the flexibility of reconfigurable networks. For example, 
on August 15, 2007 the Skype network crashed for about 48 hours, disrupting service to approximately 
200 million users due to what the company described as failures in their "self-healing mechanisms" [2j El 
O Q2J HH US |20]. We believe that this outage is indicative of a much broader problem. 

Modern reconfigurable networks have a complexity unprecedented in the history of engineering: we 
are approaching scales of billions of components. Such systems are less akin to a traditional engineering 
enterprise such as a bridge, and more akin to a living organism in terms of complexity. A bridge must 
be designed so that key components never fail, since there is no way for the bridge to automatically 
recover from system failure. In contrast, a living organism can not be designed so that no component 
ever fails: there are simply too many components. For example, skin can be cut and still heal. Designing 
skin that can heal is much more practical than designing skin that is completely impervious to attack. 
Unfortunately, current algorithms ensure robustness in computer networks through hardening individ- 
ual components or, at best, adding lots of redundant components. Such an approach is increasingly 
unscalable. 

In this paper, we focus on a new, responsive approach for maintaining robust reconfigurable networks. 
Our approach is responsive in the sense that it responds to an attack (or component failure) by changing 
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the topology of the network. Our approach works irrespective of the initial state of the network, and is 
thus orthogonal and complementary to traditional non-responsive techniques. There are many desirable 
invariants to maintain in the face of an attack. Here we focus only on the simplest and most fundamental 
invariants: ensuring the diameter of the network and the degrees of all nodes do not increase by much. 

Our Model: We now describe our model of attack and network response. We assume that the network 
is initially a connected graph over n nodes. An adversary repeatedly attacks the network. This adversary 
knows the network topology and our algorithms, and it has the ability to delete arbitrary nodes from the 
network. However, we assume the adversary is constrained in that in any time step it can only delete a 
single node from the network. We further assume that after the adversary deletes some node x from the 
network, that the neighbors of x become aware of this deletion and that the network has a small amount 
of time to react by adding and deleting some edges. This adversarial model captures what can happen 
when a worm or software error propagates through the population of nodes. Such an attack may occur 
too quickly for human intervention or for the network to recover via new nodes joining. Instead the nodes 
that remain in the network must somehow reconnect to ensure that the network remains functional. 

We assume that the edges that are added can be added anywhere in the network. We assume that 
there is very limited time to react to deletion of x before the adversary deletes another node. Thus, the 
algorithm for deciding which edges to add between the neighbors of x must be fast. 

Our Results: A naive approach to this problem is simply to 'surrogate' one neighbor of the deleted node 
to take on the role of the deleted node, reconnecting the other neighbors to this surrogate. However, 
an intelligent adversary can always cause this approach to increase the degree of some node by 9{n). 
On the other hand, we may try to keep the degree increase low by connecting neighbors of the deleted 
node as a straight line, or by connecting the neighbors of the deleted node in a binary tree. However, 
for both of these techniques the diameter can increase by 9{n) over multiple deletions by an intelligent 
adversary [3| [19] . 

In this paper, we describe a new, light-weight distributed data structure that ensures that: 1) the 
diameter of the network never increases by more than log A times its original diameter, where A is the 
maximum degree of a node in the original network; and 2) the degree of any node never increases by 
more than 3 over over its original degree. Our algorithm is fully distributed, has 0(1) latency per round 
and requires each node to send and receive O(l) messages per round. The formal statement and proof 



of these results is in Section 4.1 Moreover, we show (in Section 4.2) that in a sense our algorithm is 
asymptotically optimal, since any algorithm that increases node degrees by no more than a constant 
must, in some cases, cause the diameter of a graph to increase by a log A factor. 

The algorithm requires a one-time setup phase to do the following two tasks. First, we must find a 
breadth first spanning tree of the original network rooted at an arbitrary node. This can be done with 
latency equal to the diameter of the original network, and, with high probability, each node v sending 
O(logn) messages along every edge incident to v as in the algorithm due to Cohen [2J. The second task 
required is to set up a simple data structure for each node that we refer to as a will. This will, which 
we will describe in detail in the Section |3j gives instructions for each node v on how the children of v 
should reestablish connectivity if v is deleted. Creating the will requires O(l) messages to be sent along 
the parent and children edges of the global breadth-first search tree created in the first task. 

Related Work: There have been numerous papers that discuss strategies for adding additional capacity 
and rerouting in anticipation of failures |5l [7] [JTJ [T71 ETJ [22] • Here we focus on results that are responsive 
in some sense. Medard, Finn, Barry, and Gallager [13] propose constructing redundant trees to make 
backup routes possible when an edge or node is deleted. Anderson, Balakrishnan, Kaashoek, and Morris 
[1] modify some existing nodes to be RON (Resilient Overlay Network) nodes to detect failures and reroute 
accordingly. Some networks have enough redundancy built in so that separate parts of the network can 
function on their own in case of an attack [8]. In all these past results, the network topology is fixed. In 
contrast, our algorithm adds edges to the network as node failures occur. Further, our algorithm does not 



2 



dictate routing paths or specifically require redundant components to be placed in the network initially. 
In this paper, we build on earlier work in [3j [19] . 

There has also been recent research in the physics community on preventing cascading failures. In the 
model used for these results, each vertex in the network starts with a fixed capacity. When a vertex is 
deleted, some of its "load" (typically defined as the number of shortest paths that go through the vertex) 
is diverted to the remaining vertices. The remaining vertices, in turn, can fail if the extra load exceeds 
their capacities. Motter, Lai, Holme, and Kim have shown empirically that even a single node deletion 
can cause a constant fraction of the nodes to fail in a power- law network due to cascading failures [TO ] 1 16]. 
Motter and Lai propose a strategy for addressing this problem by intentional removal of certain nodes in 
the network after a failure begins [15_. Hayashi and Miyazaki propose another strategy, called emergent 
rewirings, that adds edges to the network after a failure begins to prevent the failure from cascading |9 . 
Both of these approaches are shown to work well empirically on many networks. However, unfortunately, 
they perform very poorly under adversarial attack. 

2 Delete and Repair Model 

We now describe the details of our delete and repair model. Let G = Go be an arbitrary graph on n 
nodes, which represent processors in a distributed network. One by one, the Adversary deletes nodes 
until none are left. After each deletion, the Player gets to add some new edges to the graph, as well as 
deleting old ones. The Player's goal is to maintain connectivity in the network, keeping the diameter of 
the graph small. At the same time, the Player wants to minimize the resources spent on this task, in 
the form of extra edges added to the graph, and also in terms of the number of connections maintained 
by each node at any one time (the degree increase). We seek an algorithm which gives performance 
guarantees under these metrics for each of the n! possible deletion orders. 

Unfortunately, the above model still does not capture the behaviour we want, since it allows for a 
centralized Player who ignores the structure of the original graph, and simply installs and maintains a 
complete binary tree, using a leaf node to substitute for each deleted node. 

To avoid this sort of solution, we require a distributed algorithm which can be run by a processor 
at each node. Initially, each processor only knows its neighbors in Go, and is unaware of the structure 
of the rest of the Go- After each deletion (forming Ht), only the neighbors of the deleted vertex are 
informed that the deletion has occurred. After this, processors are allowed to communicate by sending 
a limited number of messages to their direct neighbors. We assume that these messages are always sent 
and received successfully. The processors may also request new edges be added to the graph to form Gt- 
The only synchronicity assumption we make is that the next vertex is not deleted until the end of this 
round of computation and communication has concluded. To make this assumption more reasonable, the 
per-node communication should be 0(1) bits, and should moreover be parallelizable so that the entire 
protocol can be completed in 0(1) time if we assume synchronous communication. 

We also allow a certain amount of pre-processing to be done before the first deletion occurs. This may, 
for instance, be used by the processors to gather some topological information about Go, or perhaps to 
coordinate a strategy. Another success metric is the amount of computation and communication needed 



during this preprocessing round. Our full model is described as Model 2.1 below. 



3 The Forgiving Tree algorithm 

At a high level, our algorithm works as follows. We begin with a rooted spanning tree T, which without 
loss of generality may as well be the entire network. 

Each time a non-leaf node v is deleted, we think of it as being replaced by a balanced binary tree 
of "virtual nodes," with the leaves of the virtual tree taking v's place as the parents of v's children. 
Depending on certain conditions explained later, the root of this "virtual tree" or another virtual node 
(known as t>'s heir — this will be discussed later) takes v 's place as the child of u's parent. This is illustrated 
in figure [T] Note that each of the virtual nodes which was added is of degree 3, except the heir, if present. 



3 



Each node of Go is a processor. 

Each processor starts with a list of its neighbors in Go- 

Pre-processing: Processors may send messages to and from their neighbors. 

for t := 1 to n do 

Adversary deletes a node vt from Gt-i, forming H t . 

All neighbors of vt are informed of the deletion. 

Recovery phase: 

Nodes of Ht may communicate (asynchronously, in parallel) with their immediate neighbors. These 
messages are never lost or corrupted, and may contain the names of other vertices. 
During this phase, each node may insert edges joining it to any other nodes as desired. Nodes may 
also drop edges from previous rounds if no longer required. 
At the end of this phase, we call the graph Gt- 
end for 

Success metrics: Minimize the following "complexity" measures: 

1. Degree increase. max i<n max„ degree(t> , Gt) — degree(t>, Go) 

2. Diameter stretch. maxf <ra diam(Gi)/diam(Go) 

3. Communication per node. The maximum number of bits sent by a single node in a single 
recovery round. 

4. Recovery time. The maximum total time for a recover round, assuming it takes 1 bit no more 
than 1 time unit to traverse any edge and unlimited local computational power at each node. 

Model 2.1: The Delete and Repair Model - Distributed View. 

When a leaf node is deleted, we do not replace it. However, if the parent of the deleted leaf node 
was a virtual node, its degree has now reduced from 3 to 2, at which point we consider it redundant and 
"short-circuit" it, removing it from the graph, and connecting its surviving child directly to its parent. 
This helps to ensure that, except for heirs, every virtual node is of degree exactly 3. 

After a long sequence of such deletions, we are left with a tree which a patchwork mix of virtual 
nodes and original nodes. We note that the degrees of the original nodes never increase during the above 
procedure. Also, because the virtual trees are balanced binary trees, the deletion of a node v can, at 
worst, cause the distances between its neighbors to increase from 2 to 2 flog rf] , where d is the degree 
of v. This ensures that, even after an arbitrary sequence of deletions, the distance between any pair of 
surviving actual nodes has not increased by more than a [log A] factor, where A is the maximum degree 
of the original tree. 

Since our algorithm is only allowed to add edges and not nodes, we cannot really add these virtual 
nodes to the network. We get around this by assigning each virtual node to an actual node, and adding 
new edges between actual nodes in order to allow "simulation" of each virtual node. More precisely, 
our actual graph is the homomorphic image of the tree described above, under a graph homomorphism 
which fixes the actual nodes in the tree and maps each virtual node to a distinct actual node which is 
"simulating" it. The existence of such a mapping is a consequence of the fact that all the virtual nodes 
have degree 3, except heirs, which have degree 2 (and there are not too many of these), and will be proved 
later. Note that, because each actual node ever simulates one virtual node at a time, and virtual nodes 
have degree at most 3, this ensures that the maximum degree increase of our algorithm is at most 3. 

The heart of our algorithm is a very efficient distributed algorithm for keeping track of which actual 
node is assigned to simulate each virtual node, so that the replacement of each deleted node by its virtual 
tree can be done in O(l) time. We accomplish this using a system of "wills," in which each vertex v 
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instructs each of its children (or their "heirs") in the event of u's deletion, how to simulate the virtual 
tree replacing v, and also the virtual node v was simulating (if any). 

This will is prepared in advance, before v's deletion, and entrusted to v's children or their surviving 
heirs. An example of this is shown in figure [2} Certain events, such as the deletion of one of v's children, 
or a change in which virtual node v is simulating, may cause v to revise its will, informing the affected 
children or their surviving heirs. As shall be seen, the total size and number of messages which must be 
sent is 0(1) per deleted vertex. In addition, there is a startup cost for communicating the initial wills; 
this is O(l) per edge in the original network. 

3.1 Detailed description 

To begin with, in Table [T] we list the data kept by each real node v required for the ForgivingTree 
algorithm. We have four main classes of fields, according to the way they are used by the node. 'Current 
fields' give a nodes present configuration and status in the tree. 'Reconstruction fields' hold the data 
needed for a node to reconstruct connections when one of its neighbors gets deleted. 'Helper fields' hold 
information with regard to the helper node being simulated by this node. Each node also stores some 
special flags with regard to its helper or heir status. In the description that follows, we shall refer directly 
to these fields. 



Current fields 
parent (v) 
children (v) 
SubRT(v) 

heir (v) 


Fields having information about a node's current neighbors. 


Parent of v. 
Children of v. 

Stores the Reconstruction Tree (RT) of v minus a possible helper 
node simulated by heir(u). This tree of helper and real nodes that 
shall replace v if v is deleted. 
The heir of v. 


Helper fields 
hparent (v) 
hchildren(v) 


Fields specifying a node's role as a helper node. 


Parent of the helper node v may be simulating. 
Children of the helper node v may be simulating. 


Reconstruction fields 

nextparent (v) 
nexthparent (v) 
nexthchildren (v) 


Fields used by a node to reconstruct its connections when its neighbor 
is deleted. 


The node which will be the next parent of v. 
The node which will be the next hparent of v. 
The node(s) which will be the next hchildren of v. 


Flags 

ishelper (v) 
isreadyheir (v) 


Specifying a node's helper or heir status. 


(boolean field). True if v is simulating a helper node, false otherwise, 
(boolean field). True if v is simulating an heir in ready state, false 
otherwise (wait or deployed state). 


Table 1: The fields maintained by a node v 



At the top level, our algorithm is specified as Algorithm |3.1| : Forgiving tree. Algorithm 3.1 uses 
Algorithms 2 to 9, which will be described at the appropriate places. As referred to earlier, FORGIVING 
tree works on a tree which may be obtained from the original graph during a preprocessing phase. The 
next stage is an initialisation phase in which the appropriate data structures are setup. Once these are 
setup, the network is ready to face the adversarial attacks as and when they happen. 

3.1.1 The Initialization phase 



This phase is specified in Algorithm 3.2 : InitQ. We assume each node v has a unique identification 
number which we call ID(v). Every node in the tree initializes the fields we have listed in Table [T] In 
our descriptions if no data is available or appropriate for a field, we set it to EMPTY. Since no deletion 
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Figure 1: Deleted node v replaced by its Reconstruction Tree. The nodes in the oval are helper nodes. 
Regular helper nodes are depicted by circles and the heir helper node by a rectangle. 
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Figure 2: The leftmost column shows a small segment of the network. The RT(x) corresponding to this 
figure is shown. Every neighbor of node x stores the portion of RT(x) relevant to it. Each rectangular 
box is labelled with a neighbor and shows the portions and the value of the corresponding fields . 



has happened yet and there are no helper nodes in the system, the helper fields are set to EMPTY. The 
current fields parent(t;) and children(t;) are assigned pointers to the parent and children of v. Of course, 
if v is a leaf node children(w) is EMPTY and if v is the root of the tree parent(f) is EMPTY. 

As stated earlier, the heart of our algorithm is the system of wills created by nodes and distributed 
among its neighbors. The will of a node, v, has two parts: firstly, a Reconstruction Tree (SubRT(u)), 
which will replace v when it is deleted by the Adversary, and secondly, the delegation of u's helper 
responsibilities (if any) to a child node, heir(f). For concreteness, we initially designate the child of v 
with the highest ID as heir(u). In the event that heir(v) is deleted, its role will be taken over by its heir, 
if any. If heir(v) is a leaf when it is deleted, then v will designate its new heir to be the surviving child 
whose helper node has just decreased in degree from 3 to 2. 



Algorithm 3.5 GenerateSubRT computes SubRT(-u). If the node v has no helper responsibilities, 
as is during this phase, RT(w) is simply SubRT(?;) with a helper node simulated by heir(v) appended 
on as the parent of the root of SubRT(v). Figure [I] and Turn 1 in Fig [5] depict such Reconstruction 
Trees. If the node v has helper responsibilities RT(*y) is the same as SubRT(w). Node v uses Algorithm 



3.5 to compute SubRT(?;) as follows: All the children of v are arranged as a single layer in sorted (say, 



ascending) order of their IDs. Then a set of helper nodes - one node for each of the children of v except 
the heir are arranged above this layer so as to construct a balanced binary search tree ordered on their 
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IDs. 

The last step of the initialization process is to finalize the will and transmit it to the children. 
Each child is given only the portion of the will relevant to it. Thus, only this portion needs to be 
updated whenever a will changes. The division of RT into these portions is shown in figure |2j There are 
fundamentally two different kinds of wills : one prepared by leaf nodes who have helper responsibilities 
and the other by non-leaf nodes. Obviously, during the initialisation phase, only the second kind of will 
is needed. This is finalized and distributed as shown in Algorithm |3.6| MakeWill. The children of v 
initialize their reconstruction fields with the values from SubRT(u). If later v gets deleted these values 
will be copied to present and helper fields such that RT(f ) is instantiated. Notice that the role the heir 
will assume is decided according to whether v is a helper node or not. Since v cannot be a helper node in 
this phase, the heir node simply sets its reconstruction fields so as to be between the root of SubRT(u) 
and parent(f). In this case when RT(u) will be instantiated, the helper node simulated by heir(v) shall 
have only one child: we will say that heir(f ) is in the ready phase (explained later) and set the flag 
isreadyheir(u) to true. In the initialization phase both the isreadyheir and ishelper flags will be set to 
false. 

This completes the setup and initialization of the data structure. Now our network is ready to handle 
adversarial attacks. In the context of our algorithm, there are two main events that can happen repeatedly 
and need to be handled differently: 

3.1.2 Deletion of an internal node 



The healing that happens on deletion of a non-leaf node is specified in Algorithm 3.3 FixNodeDeletion. 
In our model, we assume that the failure of a node is only detected by its neighbors in the tree, and it 
is these nodes which will carry out the healing process and update the changes wherever required. If 
the node v was deleted, the first step in the reconstruction process is to put RT into place according 
to Algorithm |3.8| makeRT. Note that all children of v have lost their parent. Let us discuss the 
reconstruction performed by non-heir nodes first. They make an edge to their new parent (pointer to 
which was available as nextparentQ) and set their current fields. Then they take the role of the helper 



nodes as specified in RT(u) and Algorithm 3.9 MakeHelper and make the required edges and field 
changes to instantiate RT(w). 

To understand what the heir node does in this case, it will be useful here to have a small discussion 
on the states of a regular /heir node: 




Figure 3: The states of a node with respect to helper duties: Waiting, Ready and Deployed 

States of a heir/regular node: Consider a node v and its heir h. From the point of view of h, we can 
imagine h to be in one of three states which we call wait, ready and deployed. These states are illustrated 
in figure [5] For ease of discussion, let us call the helper node that a node is simulating helper(node). 
In brief, a node is considered to be in the wait state when it has no helper responsibilities, in the ready 
state when helper(node) with one child, and in the deployed state when helper(ncxie) has two children 
(which is the maximum possible). Notice that the node can be in the wait state only when v has not 
been deleted and thus, h has assumed no helper responsibilities. It only has the will of v and is in limbo 
with regard to helper duties. Now consider the case when v gets deleted. Following are the possibilities: 
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node v had no helper responsibilities: This happens when v , s original parent was not deleted. Thus, 
v could be a regular child or a heir in the wait state. On v's deletion h moves to the ready state 
and sets its flag isreadyheir to True. This is the state in which helper(/i) has only one child i.e. the 



root of SubRT(u). This happens when h executes its portion of the will of v using Algorithm 3.8 
makeRT. Note that this may not be the final state for the helper node of h, and is thus called the 
ready state. 

• node v had helper responsibilities: There are two further possibilities: 

— helper (v) had one child: This can only happen when v was a heir node in the ready state. 
Thus, v 's flags ishelper and isreadyheir were both set to True. Node h will take over the helper 
responsibilities of v and thus, in turn, h will now have one child i.e. will be in the ready state 
and will set its flags ishelper and isreadyheir to True. Notice that if v was an heir, h will 
now also take over those responsibilities, and on future deletions of w's ancestors could move 
further up the tree either as an heir in ready state or in a deployed state become a full helper 
node. 

— helper(v) had two children: Node v could be a regular child or heir, h will fully take over the 
helper responsibilities of v , and thus helper(/i) shall acquire two children and move on to the 
deployed state. Notice that previously h could have been in either wait or ready state. Since it 
is now not in the ready state, it will set its isreadyheir flag to False and ishelper flag to True. 

It is easy to see that a regular i.e. non-heir node can be in either wait or deployed state. 



Here we also define the following operation, which is used in Algorithms 3.4 and |3.8 



bypass(x): Precondition: |hchildren(x)| = 1 i.e the helper node has a single child. Operation: Delete 
helper(x) i.e. hparent(x) and hchildren(x) remove their edges with x and make a new edge between 
themselves, hparent(x) <— EMPTY] hchildren(x) <— EMPTY. 

We can now easily see how the heir of v, h takes part in the reconstruction according to Algorithm 



3.8: makeRT. Node h can be either in wait state or ready state. If it is in the wait state it simply takes 



its helper responsibilities according to Algorithm 3.9: MakeHelper, as in turn 1 of figure [5] . Note 



that here h checks if it has moved to the ready state and sets its isreadyheir flag accordingly. If h was 
already in ready state, it relinquishes its present helper role and moves on to the new helper role. To 
relinquish its present role, node h intimates hparent(Zi) and hchildren(Zi), and together they accomplish 
this as specified by the operation bypass(/i). Turn 2 in Fig [5] illustrates this. 

Once RT(u) is in a place, there may be a need for the parent of v to recompute its will. This happens 
only when v did not already have a helper role or equivalently when heir(i>) moves to a ready state. Lines 
[2] to [6] of Algorithm 3.3 deals with this situation. Node parent(u) simply replaces v by heir(v) in its will 



and retransmits it. At the end of this healing process, the children of the deleted nodes check if they 
need to leave the second kind of will, which we call a LeafWill. This will is required only for those nodes 
which are leaves in our tree and have virtual responsibilities. Since they have no children to take over 
their helper responsibilities they leave this responsibility to their parent. We will discuss this in greater 
detail in the next section. 

3.1.3 Deletion of a leaf node 

If the adversary removes a leaf node from the system, the healing is accomplished by its neighbors as 



specified in Algorithm 3.4 FixLeafDeletion. Let v be the deleted leaf node and p be its parent. Let 
us consider the simple case first. This is when the deleted node had no helper responsibility. This also 
implies its original parent did not suffer a deletion. Node p simply removes v from the list of its children 
and then recomputes and redistributes its will. 
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(a) helper(f) is ancestor of v. (b) w and helper(ui) share a neighbor. 




(c) 2 and helper(z) do not share neighbors. (d) z is an heir in Ready state. 

Figure 4: Various cases of Leaf deletions 

Now, consider the situation where the deleted node had helper responsibilities. In this case, the one node 
whose workload has been reduced by this deletion is p. Using Algorithm |3.7| MakeLeafWill v hands 
over the list of its helper responsibilities to p. Here, a special case may arise when v is simulating a helper 
node which has v itself as one of its hchildren. Recall that parent (v) is v 's ancestor closest to v in the 
tree. This implies that parent(u) = hparent(v) = p. The only thing that p needs to do if v is deleted is to 
remove v from its hchildren and add itself (for consistency). This is the will conveyed by v to p. When 
v is deleted, p simply updates its helper fields. For other cases, v simply sends its helper fields to p to 
be copied to p's helper reconstruction fields. In this situation, when v is actually deleted the following 
happens: the helper node that p is simulating is deleted and bypassed by the bypass operation defined 
earlier. Node p now simulates a new helper node that has the same helper responsibilities previously 
fulfilled by v. In case the deleted leaf node was itself an heir in ready state, p detects this and sets its flags 
accordingly. Again at the end of the reconstruction, the leaf nodes reconstruct their wills. An example 
of such a leaf deletion is the deletion of node d at Turn 3 as shown in figure [5] 

Important Note: When implementing the pseudocode for Algorithm |3.6[ Make Will, it is important 
to bear in mind that when RT(n) is being updated due to a node deletion, most of SubRT(u) will be 
unchanged. In fact, only O(l) nodes will need to have their fields updated. These can be found and 
updated more efficiently by a more detailed algorithm based on case analysis, which we defer to the full 
version of this paper. 



1 


Given a tree T(V, E) 


2 


Init(T). 


3 


while true do 


4 


if a vertex x is deleted then 


5 


if children(x) is EMPTY then 


6 


FixLeafDeletion(x) 


7 


else 


8 


FixNodeDeletion (x) 


9 


end if 


10 


end if 


11 


end while 



Algorithm 3.1: Forgiving tree: The main function. 
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Require: each node of T has a unique ID 


1 


for each node v G T do do 


2 


children (v) <— children of v. 


3 


parent(v) <— if v is root of T then EMPTY else parent of v. 


4 


isreadyheir(-u) <— false. 


5 


ishelper(f ) <— false. 


6 


hparent(u) <- EMPTY. 


7 


hchildren(w) «- EMPTY. 


8 


heir(u) <— if v is a leaf node then EMPTY else child of ij with highest ID. 


9 


SubRT(w) <- generateSubRT(w). 


10 


MakeWill(w, SubRT(t;)). 


11 


end for 


Algorithm 3.2: Init(T): initialization of the Tree T 



1 


MAKERT(children(t;) , parent(f )) . 


2 


let h = heir(u).Let p = parent(u) 


3 


if isreadyheir(Zi) = true then 


4 


hparent (/i) replaces v by h in SubRT (hparent (/i)). 


5 


Make Will (hparent (h) , SubRT (hparent (h))). 


6 


end if 


7 


for each node y £ children(w) do 


8 


if children(w) is EMPTY then 


9 


MakeLeafWill(w) . 


10 


end if 


11 


end for 


Algorithm 3.3: FixNodeDeletion(v): Self-healing on deletion of internal node 
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hchildren(2^) ^ — nexthchildren(z). 


14 


if hchildren(z) 1=1 then 

1 V / 1 


15 


isreadyheir(z) = true 


16 


end if 


17 


end if 


18 


for each node y G children (v) do 


19 


if children(t;) is EMPTY then 


20 


MakeLeafWill(?j) . 


21 


end if 


22 


end for 



Algorithm 3.4: FixLeafDeletion(-u): Self-healing on deletion of leaf node 



1: Let Lset be a set of vertices representing all members of children(w), and I set be another set of 

vertices, representing all members of children(u) except the one with the highest ID. 
2: Arrange Lset in ascending order of their IDs. 

3: Using the arranged Lset as leaves and lset as the internal nodes construct a Balanced Binary Search 

Tree SubRT ordered on the nodes ID. 
4: return SubRT 

Algorithm 3.5: GENERATESuBRT(children(w)): Computes the Reconstruction Tree (RT) of v minus 
a possible helper node simulated by heir(w). 
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1 


Let p = parent (v). Let rv be root of SubRT (v). 


2 


for each node y £ children ) do 


3 


let ly be the leaf vertex representing y in SubRT(v). Let hy be the internal node in SubRT 




representing y. 


4 


If hy is /y's parent in SubRT(t>) then nextparent(y) <— parent of hy in SubRT else nextparent(y) <— 




parent of ly in SubRT. 


5 


if y / heiriv) then 


6 


nexthchildren(y) <— children of hy in SubRT. 


7 


nexthparent(y) <— parent of /iy in SubRT. 


8 


else 


9 


if ishelper(f ) = true then 


10 


nexthchildren(y) <— hchildren(v). 


11 


nexthparent(y) <— hparent(w). 


12 


nexthparent(rw) <— p. 


13 


else 


14 


nexthchildren(y) <— rt>. 


15 


nexthparent(y) <— p. 


16 


nexthparent(rv) <— y. 


17 


end if 


18 


end if 


19 


end for 


Algorithm 3.6: MAKEWiLL(t> , SubRT(w)): Makes and distributes the will of v 


1 


let z = parent (v). 


2 


if z = hparent(w) then 


3 


nexthparent(z) <— hparent(w). 


4 


nexthchildren(z) <— hchildren(z)/{w} U {z}. / / z will take on itself as a child of its helper node. 


5 


else 


6 


nexthparent(z) <— hparent(w). 


7 


nexthchildren(z) <— hchildren(w). 


8 


end if 


Algorithm 3.7: MakeLeafWill(u): Leaf node leaves a will for its parent. 


1 


for each node x 6 children (v) do 


2 


if isreadyheir(x) = true then 


3 


bypass(x). // hparent(x) and hchildren(x) bypass x and connect themselves. 


4 


MakeHelper(x). 


5 


else 


6 


x makes edge between itself and nextparent(x). 


7 


parent(x) <— nextparent(x). 


8 


MakeHelper(x). 


9 


end if 


10 


end for 



Algorithm 3.8: makeRT (children (v), parent (v)): Replace the deleted node by its RT 
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1: v makes edges between itself and nexthchildren(u), and nexthparent(v). 

2: hparent(f) <— nexthparent(v). 

3: hchildren(u) <— nexthchildren(i;). 

4: ishelper(u) = true. 

5: if |hchildren(u)| = 1 then 

6: isreadyheir(-u) = true. / / Only an 'unemployed' heir has a single child. 
7: end if 

Algorithm 3.9: MakeHelper(-u): v takes over helper node responsibilities 

4 Results 

4.1 Upper Bounds 

Let T be the original tree, let D be its diameter, and let A be its maximum degree. 
Theorem 1. The Forgiving Tree has the following properties: 

1. The Forgiving Tree increases the degree of any vertex by at most 3. 

2. The Forgiving Tree always has diameter O(Z) log A). 

3. The latency per deletion and number of messages sent per node per deletion is 0(1); each message 
contains 0(1) bits and node IDs. 

Proof. Parts [T] and [3] follow directly by construction of our algorithm. For part[l] we note that for a node 
v, any degree increase for v is imposed by its edges to hparent(f) and hchildren(f ). By construction, 
node v can play the role of at most one helper node at any time and the number of hchildren is never 
more than 2, because the reconstruction trees are binary trees. Thus the total degree increase is at most 
3. Part [3] also follows directly by the construction of our algorithm, noting that, because the virtual 
nodes all have degree at most 3, healing one deletion results in at most 0(1) changes to the edges in 
each affected reconstruction tree. In fact, the changes to RT(u>) for an affected node w do not require 
new information, which allows these messages to be computed and distributed in parallel. We leave the 
details to the full version due to space constraints. 

We next show Part [2] that the diameter of the Forgiving Tree is always 0(D log A). We assume that 
T is rooted with some root r selected arbitrarily and let p{v) be the parent of v in T. Fix any node v\ 
in the tree and let P = . . . , be the path in T from the node v\ to the root V£ = r. Let T 1 be the 
Forgiving Tree data structure after some fixed number of deletions. To simplify notation, if a node v has 
not been deleted in T 1 , we will let root of RT(w) be v by default. We construct a path P' through T 1 from 
v± to the root of RT{r) as follows. For all i from 1 to £, P' connects the root of RT(uj) to the root of 
RT(vj+j) for the smallest j > 1 such that RT(vi+j) exists. We note that if any nodes remain in T', then 
RT(r) must still exist and so P' connects v\ to the root of RT(r). We further note that the length of P' 
is no more than a factor of log A + 1 greater than the length of P. To see this, consider the length of 
the path connecting the root of RT(wj) to the root of RT(wj + j) for the smallest j > 1 such that RT(uj +J ) 
exists. Such a path can have length at most equal to the log of the number of children of Vi+j in T, a 
quantity that can be at most log A + 1. The length of the path between the roots will not increase even 
when nodes in KT(vi + j) are deleted. 

Now consider any pair of nodes x and y in T 1 . We know from the above that x and y are connected to 
the root of RT(r) in T' by paths of length at most h ■ (log A + 1), where h is the height of T. We further 
know that the diameter of T, D, is at least h. It follows directly that the diameter of the Forgiving Tree 
is never more than 0(1) log A). □ 
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4.2 Lower Bounds 

Theorem 2. Consider any self-healing algorithm that ensures that: 1) each node increases its degree by 
at most a, for some a > 3; and 2) the diameter of the graph increases by a multiplicative factor of at 
most (3. Then for any positive A, for some initial graph with maximum degree A, it must be the case that 
a 2 ^ 1 > A. 

Proof. Let G be a star on A + 1 vertices, where x is the root node, and x has A edges with each of the 
other nodes in the graph. Let G' be the graph created after the adversary deletes the node x. Consider 
a breadth first search tree, T, rooted at some arbitrary node y in G' . We know that the self-healing 
algorithm can increase the degree of each node by at most a, thus every node in T can have at most 
a + 1 children. Let h be the height of T. Then we know that 1 + (a + 1) Yli=o a% — A. This implies that 
a h+i > A for a > 3, or /i + 1 > log Q A. Since the diameter of G is 2, we know that (3 = h/2, and thus 
2(3 + 1 > logo, A. Raising a to the power of both sides, we get that a 2/3+1 > A. □ 

We note that this lower-bound compares favorable with the general result achieved with our data struc- 
ture. The Forgiving Tree can be modified so that it ensures that 1) the degree of any node increases by 
no more than a for any a > 3; and that the diameter increases by no more than a multiplicative factor 
of f3 < 2 log a A + 2. Thus we can ensure that a(V2)(/3-i) < a. 

5 Conclusion 

We have presented a distributed data structure that withstands repeated adversarial node deletions by 
adding a small number of new edges after each deletion. Our data structure ensures two key properties, 
even when up to all nodes in the network have been deleted. First, the diameter of the network never 
increases by more than O(logA) times its original diameter, where A is the maximum original degree 
of any node. For many peer-to-peer systems, A is at most polylogarithmic, and so the diameter would 
increase by no more than a O(loglogn) multiplicative factor. Second, no node ever increases its degree 
by more than 3 over its original degree. 

Several open problems remain including the following. Can we protect other invariants? For example, 
can we design a distributed data structure to ensure that the stretch between any pair of nodes increases 
by no more than a certain amount? Can we design algorithms for less flexible networks such as sensor 
networks? For example, what if the only edges we can add are those that span a small distance in the 
original network? Finally, can we extend the concept of self-healing to other objects besides graphs? For 
example, can we design algorithms to rewire a circuit so that it maintains its functionality even when 
multiple gates fail? 
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Turn 1: Adversary a- 
deletes v b 



P 



b -- da --- f 
a' b d' il d' f g h 



Vertices a through h 
take over virtual nodes 
in RT(v). h is v's heir 
and connects to both p 
and d. Note that the real 
graph now contains a 
cycle, (b, c, d) 



Turn 2: Adversary 
deletes p 



Turn 3 : Adversary 
deletes d 




Turn 4: Adversary 
deletes h 



a-b- 





Vertices h, i, j, and k 
take over virtual nodes 
in RT(p). h takes over 
the helper role of v in 
RT(p). d attaches to i. 
k is p's heir and connects 
to both h and parent(p). 



The virtual node of c is 
bypassed and c takes 
over the helper role of d. 



Vertices m, n and o 
take over virtual nodes 
of RT(h). o is heir of h 
and takes over h' s helper 
role. Note that since number 
of children of h was not a 
power of 2, not all the leaves 
of RT(h) are at the same 
depth. 



a-b 




The network after all the deletions and 
self healing. 



Figure 5: An illustrative sequence of deletions and healings. 
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